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DYNAMIC RANGE COMPRESSION OF OUTPUT CHANNEL DATA OF 
AN IMAGE SENSOR 

FIELD OF THE INVENTION 

This invention relates to dynamic range 
compression of output channel data of an image 
sensor. This invention is particularly useful for, 
but not limited to, dynamic range compression of 
channel data representing colors generated by solid- 
state image sensors such as Complementary Metal Oxide 
Semiconductor (CMOS) image sensors that are 
particularly prone to generating noise. 

BACKGROUND ART 

Various types of solid-state image sensors are 
in use today, including charge-coupled device (CCD) 
image sensors and complimentary metal -oxide 
semiconductor CMOS image sensors. 

CMOS image sensors typically utilize an array of 
active pixel image sensors and a row of register of 
correlated double- sampling (CDS) circuits or 
amplifiers to sample and hold the output of a given 
row of pixel image sensors of the array. Each active 
pixel image sensor of the array of pixels typically 
contains a pixel amplifying device (usually a source 
follower) . 

Current CMOS image sensors have inferior image 
performance compared to CCD image sensors. This 
inferior image performance is due to inherent Fixed 
Pattern Noise (FPN) , limited dynamic range (about 
72dB) which is reduced, in part, by FPN, and low fill 
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factor (the ration of photo-detector areas to total 
of the APS pixel circuitry) which results in lower 
sensitivity. A digital image processor is therefore 
necessary in order to improve the image quality of 
CMOS image sensors . 

Several image enhancement algorithms have been 
developed for digital cameras that use solid state 
image sensors. However, because these algorithms are 
developed and optimized for CCDs, they do not achieve 
the same image enhancement for cameras using CMOS 
image sensors . 

A major goal of image enhancement is to process 
an original image in a manner such the processed 
image is more suitable for a desired application. 
Image details that are blurred or corrupted are 
enhanced by removal of perceived noise from the 
image. However, by merely removing perceived noise 
often leads to removal of important image information 
along with noise. 

Most of the image enhancement techniques operate 
on a color interpolated version of the image. The 
color interpolated version is generated from a brayer 
pattern. Each pixel location of the image sensor has 
the intensity level of only one of three color 
components Red, Green or Blue that are used to 
provide respective R, G, B output channels. The 
brayer pattern is then color interpolated such that 
each pixel's, missing color components are 
approximated to give each missing color. However, 
color interpolation inherently introduces noise into 
the image. A major requirement of image enhancement 
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is therefore to accurately distinguish between true 
image details and false image details such as noise. 

There are many image enhancement algorithms that 
include a function to expand the dynamic range of the 
image. An attractive way is described in Hansoo Kim 
& al., "Digital Signal Processor with Efficient RGB 
Interpolan and Histogram Accumulation", IEEE Trans. 
On Consumer Electronics, vol 44, No . 4 , November 1998 
pp 1389-1395. In this paper, a processor controls a 
gamma slope to extract a set of feature data from an 
output image. The processor calculates then the 
numbers of pixels of luminance values in the bright 
range, in the middle range and in the dark range. 
Another method presented in "Adaptive Gamma 
Processing of the video Cameras for the expansion of 
the dynamic range", IEEE Trans. on Consumer 
Electronics vol. 41, No. 3, August 1995, pp 555-562 
makes a histogram of the luminance level in a frame 
and expands the dynamic range by controlling the 
slopes of knee compensation. 

Unfortunately, prior art dynamic range 
compression methods do not process pixel values to 
suit the behaviour of the human vision in which lines 
and edges are generally of importance. Prior art 
dynamic range compression methods do not ideally 
compensate for changing frequency characteristics 
with mean luminance which can be important because 
the nature of the noise present in the image changes 
significantly with mean luminance. At low values of 
mean luminance there is a significant proportion of 
high frequency noise caused by random photon events. 
At higher mean luminance values the signal to noise 
ration improves as the proportion of random photon 
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events decreases, making detection of high frequency 
component far less error prone. Hence it would be 
useful for a dynamic range compression method to 
adopt the strategy of changing from object detectors 
5 at low light levels to feature detectors in bright 

light. This is due to the change in frequency 
characteristics in visual cells in the retina which 
act as low pass filters at low light levels and 
become bandpass filters as light intensity increases. 

10 

In this specification, including the claims, the 
terms 'comprises', 'comprising' or similar terms are 
intended to mean a non-exclusive inclusion, such that 
a method or apparatus that comprises a list of 
15 elements does not include those elements solely, but 

may well include other elements not listed. 

SUMMARY OF THE INVENTION 

2 0 According to one aspect of the invention there 

is provided a method for dynamic range compression of 
output channel data from an image sensor comprising 
an array of sensor cells, the method comprising the 
steps of: 

2 5 selecting a window in the channel data, the 

window having a reference pixel value and a 
plurality of nearby pixel values, said reference 
pixel value originating from a reference cell 
that is one of the sensor cells and said nearby 

3 0 pixel values originating from said sensor cells 

that are in close proximity to said reference 
cell; 

multiplying said pixel values in the window 
by a respective weight value to provide weighted 
3 5 pixel values; 
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adding said weighted pixel values to 
provide a convolution value; 

providing a dynamic range compression value 
for said window from a selected one of said 
pixel values and said convolution value; and 

assigning the dynamic range compression 
value to a selected pixel location comprising 
part of an image. 

Preferably, the step of providing may be 
effected by dividing the said selected one of said 
pixel values by the convolution value to provide said 
dynamic range compression value. Preferably, said 
selected one of said pixel values may be said 
reference pixel value. 

Suitably, the widow may correspond only to a 
chrominance channel . 

Preferably, the above steps may be repeated for 
other windows in said channel. 

Suitably, the above steps could be repeated so 
that the step of assigning thereby assigns a said 
dynamic range compression value to every pixel 
location providing a luminance color spectrum for 
said image 

The reference cell may be suitably a centrally 
located sensor cell in an array of said sensor cells 
providing said nearby pixel values. 

Preferably, said step of multiplying may be 
further characterised by the respective weight value 
having the effect of Gaussian filtering. The step of 
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multiplying may be further characterised by all said 
pixel vales in the window being multiplied by a said 
respective weight value. 

Suitably, said dividing may have the effect of 
non-linear filtering said pixels values in said 
window . 

Preferably, the steps of adding, providing and 
assigning can be effected by: 

*ref 
A + W®I 

where i ^ is the reference pixel value; I is a 
matrix representing the pixel values in the window for 
the luminance channel; W is a weight matrix 
representing strengths of connections for a 
corresponding sensor cell in the matrix I; and A is 
matrix containing decay values of the corresponding 
cell in matrix I; and ® is a convolution function. 

According to another aspect of the invention 
there is provided a digital camera comprising: 

an image sensor comprising an array of 
sensor cells; and 

a Channel processing circuit coupled to 
receive channel output data from the image 
sensor, 

wherein in use the Channel processing 
circuit selects a window in the channel data, 
the window having a reference pixel value and a 
plurality of nearby pixel values, said reference 
pixel value originating from a reference cell 
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that is one of the sensor cells and said nearby 
pixel values originating from said sensor cells 
that are in close proximity to said reference 
cell; 

and thereafter assigns a dynamic range 
compression value to a selected pixel location 
comprising part of an image, the dynamic range 
compression value determined from the steps of: 

multiplying said pixel values in the window 
by a respective weight value to provide weighted 
pixel values; 

adding said weighted pixel values to 
provide a convolution value; and 

dividing a selected one of said pixel 
values by said convolution value. 

The camera may be further characterised by one 
or more of the above method features or steps. 

BRIEF DESCRIPTION OF THE DRAWINGS 

In order that the invention may be readily 
understood and put into practical effect, reference 
will now be made to a preferred embodiment as 
illustrated with reference to the accompanying 
drawings in which: 

Fig. 1 is a block diagram of a camera in 
accordance with the present invention; and 

Fig. 2 illustrates a schematic block 
diagram of a channel processing circuit 5 that 
is part of the camera of Fig. 1; 
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Fig. 3 is a flow diagram of a method for 
providing dynamic range compression on output 
channel data from pixel values generated by a 
sensor of the camera of Fig. 1; and 

5 

Fig. 4 is a visual representation of a 
window and corresponding weight values used in 
the method of Fig. 3. 

10 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT OF 

THE INVENTION 

With reference to Fig. 1 there is illustrated a 
block diagram of a digital camera 1. Camera 1 

15 includes an image sensor 2, typically a CMOS image 

sensor, having a Bayer pattern output or raw image 
output coupled to an interpolation module 3 . As will 
apparent to a person skilled in the art, the image 
sensor 2 comprises an array of sensor cells that are 

2 0 arranged to be sensitive to a selected one of the 

primary colors (red, green and blue) . The color 
interpolation module 3 has three channel outputs 
these being a Red interpolated output Ri, a Blue 
interpolated channel output Bi and a Green 

2 5 interpolated channel output Gi . The three channel 

outputs Ri, Bi and Bi are coupled to, and provide 
interpolated color output channel data to, respective 
inputs of a color correction circuit 4. Outputs of 
the color correction circuit 4 are a Red corrected 

30 channel output Rc, a Green corrected channel output 

Gc and Blue corrected channel output Be. Further, 
the outputs of the color correction circuit 4 are 
coupled to, and provide corrected color output 
channel data to, respective inputs of a channel 

35 processing circuit 5. 
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Outputs of the channel processing circuit 5 are 
a Dynamic Luminance Compressed DLC channel output and 
color space converted and filtered red and blue 
outputs C RFIL , C BF1L , these outputs of circuit 5 are 
coupled to inputs of a memory unit 6. 

There is also a control module 7 coupled to a 
display unit 8 in the form of a Liquid Crystal 
Display for displaying displaying an image comprising 
processed color sampled signals provided at the 
outputs of circuit 5 and stored in the memory unit 6. 

The camera 1 also includes a user input module 9 
coupled to provide command signals to the control 
module 7. There is also an output port 10 coupled by 
a common bus 2 0 to the control module 7 and the 
memory unit 6 is coupled to the output port 10 by a 
bus 12. Further, the sensor 2, color interpolation 
module 3, color correction circuit 4, channel 
processing circuit 5 and memory unit 6 are coupled to 
the control module by the common bus 20. 



Referring to Fig. 2, there is illustrated a 
schematic block diagram of the channel processing 
circuit 5. The channel processing circuit 5 includes 
a color space conversion circuit 21 with inputs 
coupled to the Red corrected channel output Rc, the 
Green corrected channel output Gc and the Blue 
corrected channel output Be. in use, data on each 
channel output Rc,Gc,Bc is converted by circuit 21 
into luminance and chrominance output channel data 
(defined by international standards) provided at a 
respective luminance channel Y and chrominance 
channels C R , C B . 
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The luminance channel Y is coupled to an input 
of a dynamic range compression circuit 22 and the 
chrominance channels C R/ C B are coupled to inputs of 
5 a median filter 23. The median filter is a non- 

linear filter that replaces each grey level pixel 
value, associated with one of the sensor cells, in 
one of the chrominance channels with a corresponding 
median value. For each replaced grey level pixel 
10 value, the median value is obtained by calculating 

the median grey levels of nearby pixel values 
corresponding to the sensor cells that are in close 
proximity to the sensor cell associated with the 
replaced grey level pixel value. 

15 

An output from the dynamic range compression 
circuit provides the Dynamic Luminance Compressed DLC 
channel output and outputs from the median filter 
provide the color space converted and filtered red 
20 and blue outputs C RFIL , C BFIL . 

In this embodiment the dynamic range compression 
circuit 22 basically includes a processor and 
associated memory and therefore dynamic range 

2 5 compression is implemented in firmware. The firmware 

is based on a Cellular Neural Network CNN structure. 
Neural Networks as well known in the art and comprise 
a two dimensional array of mainly identical cells 
that satisfy two properties, these being: (a) that 

30 most interactions are local; and (b) that all state 

variables are continuous valued signals. Further 
details of Neural Networks are described in L. 0. 
Chua, L . Yang, " Cellular Neural Networks: Theory", 
IEEE Trans, on Circuits and Systems, vol. 35, pp. 
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1257-1272, Oct. 1988. This article is incorporated 
into this specification by reference. 

In use, images stored in the memory unit that 
are supplied from the output channels DLC, C RFIL and 
C bfil are displayed upon the display unit 8 in 
response to suitable commands provided from the user 
input module 9 . Such sampled signals are obtained by 
the control module 7 controlling the Sensor 2, color 
interpolation module 3, circuits 4,5 and memory unit 
6 and display unit 8. Further, if a user wishes to 
transfer images stored on the memory unit 6, then by 
suitable commands effected by the user input module 
9, images can be loaded onto a removable memory 
inserted into the output port 10. 

Referring to Fig. 3 there is illustrated a 
method SI 00 for providing dynamic range compression 
on processed output channel data from the image 
sensor 2 comprising an array of sensor cells. The 
image sensor 2 provides an output channel of data in 
a Bayer Pattern PB that is processed by the color 
interpolation module 3, color correction circuit 4 
and color space conversion circuit 21. in this 
embodiment, output channel data of processed pixel 
values y p . x for the sensor cells of image sensor 2 
associated with the luminance channel Y are processed 
by the method SI 0 0 in the dynamic range compression 
circuit 22. Upon control module 2 sending a start 
command the method S100 effects a start step S110 and 
then a setting step S120 that sets a counter K to 1 . 
A selecting a window step 13 0 in the channel data is 
then effected in which part of the channel data for 
the luminance channel Y stored in circuit 22 is 
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selected as a window. The window has an associated 
reference pixel value and a plurality of nearby pixel 
values , the reference pixel value originates from a 
processed output of a reference cell that is one of 
5 the sensor cells and the nearby pixel values 

originate from a processed output of sensor cells 
that are in close proximity to the reference cell. 
More specifically, the reference cell is a centrally 
located cell in an array of the sensor cells that 
10 provide the nearby pixel values. 

The circuit 22 then effects a step S140 of 
multiplying each pixel value in the window by a 
respective weight value to provide weighted pixel 
15 values. This has the effect of Gaussian filtering 
the pixel values due to the weights W(x, y) being of a 
Gaussian form: 

W(x,y) = Cexp[-(x 2 + y 2 )/2a 2 ] - (1) 

20 

where C is a constant) ; a is a standard deviation; 
x and y are the indexes of the pixels. 

A step S150 of adding the weighted pixel values 
2 5 to provide a convolution value F is then effected. 

In this embodiment the window represents a 3 by 3 
array therefore there are eight nearby pixels in the 
window . 

30 The circuit 22 then effects a step S160 of 

providing a dynamic range compression value DRCV for 
the window from the reference pixel value and the 
convolution value. In this regard, the dynamic range 
compression value is determined by dividing the 
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reference pixel value by the convolution value as 
described in equation (2). 

DRCV = 1 ref — — (2) 

A + W0I 

5 

where i ^ is the reference pixel value; I is a 

matrix representing the processed pixel values y pix in 
the window identified by counter K for sensor cells 
associated with the luminance channel Y; W is a 

10 weight matrix representing strengths of connections 

for a corresponding cell in matrix I; and A is matrix 
containing constant decay values of the corresponding 
cell in matrix I; and ® is a convolution function. 
The dividing of the reference pixel value by the 

15 convolution value has the effect of non-linear 

filtering the pixel values y pix in the window. 

At a step S170 an assigning of the dynamic range 
compression value DRCV to a selected pixel location 
2 0 comprising part of the image that is stored in the 

memory unit 6. A test step S180 then determines if 
there are any more windows to be processed and if 
there are more windows in the channel data then the 
counter K is incremented at a step S2 00 and steps 

2 5 SI 3 0 to SI 80 are repeated for a new window indexed by 

counter K. Thus, every pixel location in the 
luminance color spectrum for the image is assigned a 
dynamic range compression value DRCV. 

3 0 In Fig. 4, a visual representation of a window 

in the channel data 40 and corresponding weight 
values 45 used in the method S100 is illustrated. 
The window represents a 3 by 3 array of pixel values 
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I i,i' I i,2' I i, 3 ' I 2,i' I 2,2' I 2, 3 ' I 3,i' I 3,2' I 3,3 whereby the 
reference pixel is I 2 2 . The corresponding weight 
values 45 are in a weight matrix W with elements 

W 1,1' W 1,2' W 1,3' W 2,1' W 2,2' W 2,3' W 3,1' W 3,2' W 3,3- TtlUS USin 9 

equation (2) the method S100 (and assigning all 
elements of matrix A to a value of 1) , a dynamic 
range compression value DRCV for this window with 
reference pixel I 22 is calculated as follows: 

lv 

1+ h.W.l +h,2 W U2 +/l. 3 Wi.3 +h,lW 2yl +I 2 , 2 W 2t2 +/ 2 , 3 W 2 , 3 +h,lW 3J +/ 3 , 2 W 3 , 2 +/ 3 ,3^,3 



Advantageously, the present invention 

compensates for changes in frequency characteristics 
acting in the same way visual systems behaves. It is 

15 well known in biological vision that changing 

frequency characteristics with mean luminance is 
important because the nature of the noise present in 
the input changes significantly with mean luminance. 
At low values of mean luminance there is 

2 0 a significant proportion of high frequency noise 

caused by random photon events . At higher mean 
luminance values the signal to noise ratio improves 
as the proportion of random photon events 
decreases, making detection of high frequency 

2 5 component far less error prone. This invention 

processes the output channel data by the circuit 
22 effectively changing from an object detector for 
low light level pixeel values to a feature detector 
for bright light level pixel values. This is due to 

3 0 the change in frequency characteristics in visual 

cells, which act as low pass filters at low light 
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levels and become bandpass filters as light intensity 
increases . 

The detailed description provides preferred 
5 exemplary embodiments only, and is not intended to 

limit the scope, applicability, or configuration of 
the invention. Rather, the detailed description of 
the preferred exemplary embodiments provide those 
skilled in the art with an enabling description for 

10 implementing preferred exemplary embodiments of the 

invention. It should be understood that various 
changes may be made in the function and arrangement 
of elements without departing from the spirit and 
scope of the invention as set forth in the appended 

15 claims. 



